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Abstract. (N x AQ-matrix is called additive when its elements are pair-wise 
sums of N real numbers a^. For a quadratic binary functional with an additive 
connection matrix we succeeded in finding the global minimum expressing it 
through external parameters of the problem. Computer simulations show that en- 
ergy surface of a quadratic binary functional with an additive matrix is complicate 
enough. 



1 Introduction 

In the present paper we analyze the classic problem of discrete mathematics that is 
minimization of a quadratic functional depending on the great number N of binary 
variables sf. 

E(s) = = J2 Ji r^ s i — * min ' "* = ±L 

This problem arises in a lot of scientific fields of knowledge beginning from physics of 
magnetic materials and neural networks up to analysis of results of physical experiments 
and logistics. Usually the connection matrix J = (Jij)f is supposed to be symmetric 
one with zero diagonal elements: = Jji, J a = 0. The state of the system as a whole 
is given by A^-dimensional vector s = (si, S2, Sj\r)- Such vectors will be called 
configuration vectors or simply configurations. The characteristic E(s) that has to be 
minimized will be called the energy of the state, and the configuration providing the 
global minimum of the functional (1) will be called the ground state. 

In general the number of local minima of the functional (1) is exponentially large. 
Practically all minimization algorithms guarantee finding of a local minimum only. The 
exceptions are very rare and, as a rule, they are relied on specific properties of the con- 
nection matrix ifll, ll2l. The most widespread is the random minimization [ 1 1. According 
this algorithm the spin dynamics is started from a random configuration. In randomized 
order the states of dissatisfied spins are changed. As a result the dynamic system step 
by step falls into the nearest local minimum. We used just the random minimization in 
our computer simulations (Section 3). 

Very little is known about properties of the energy surface of the functional (1), 
namely, about the number and the structure of the set of local minima, about the ground 



state and the probability to find it and so on. In fact there is only one nontrivial con- 
nection matrix for which the ground state of the functional (1) can be indicated exactly. 
This is the Hebb matrix in the case when the value of the loading parameter a = M/N 
is small: a < 0.07 [3|. Then the global minimum of the functional (1) is achieved at 
any of M random patterns. 

Due to discrete character of the problem its theoretical analysis is very rare. From 
recent results let us point out the papers [4|, [5|, where the authors succeeded in con- 
necting the depth of the local minimum with the probability of its random finding, and 
also described some characteristics of the energy surface. 

In our work we introduce a class of additive matrices whose elements are pair-wise 
sums of a set of predetermined numbers ac 

Jij = (1 - 5y)(oi + Oj), i,j = l,...,N, where {ajf G R 1 (2) 

and 5ij is the Rronecker delta symbol. 

The additive matrices generalize a special class of Hebb's matrices analyzed in [6 1. 
For the functional (1) with the connection matrix (2) the ground state can be obtained 
exactly. We succeeded in presentation additive matrices in the form when the depen- 
dence of the ground state on external parameters of the problem can be described ana- 
lytically. When the ground state is known, interesting results can be obtained with the 
aid of computer simulation. In the next Section we present the theory relating to the 
problem. In Section 3 we give the results of computer simulations. 



2 The Ground State 



1. It can be verified directly that for an additive matrix (2) the value of the functional 
(1) is equal to 

E(s) = (e,a)-(B,e)(s,a) j (3) 

Here a = (ai, et2, .., a at) is TV-dimensional vector whose coordinates are a,-, and e = 
(1, 1, 1) is the "bisector" of the principal orthant of the space R N . From minimiza- 
tion of the functional (3) one can pass to maximization of the functional 

F(s) = (s, e)(s, a) — ► max. (4) 

Let us denote by E k the class of all configurations s for which exactly k coordinates 
are equal 

S k = {s: (s, e) = N - 2k} , k = 0, 1, N. 

The class Sk consists of configurations. For all these configurations the first mul- 
tiplier in the expression (4) takes the same value N — 2k. Consequently, to maximize 
(4) among configurations from the class E k , it is sufficient to find a vector s G S k 
maximizing the scalar product (s, a). This problem is not so difficult (see item 3). 



2. Suppose, we can find the vector s maximizing the scalar product (s, a) in the 
class E k . Let us denote this vector as s(k), and let the value of the functional (4) for 
this vector be F k — F(s(k))\ 

F k = (N- 2fc)(s(fc),a) = maxF(s), k = 0,1,. .,7V. 

When finding all these vectors s(fc) (k = 0, 1, .., N), it is easy to find the global maxi- 
mum of the functional (4), since the functional reaches its maximal value on one of the 
vectors s(k). 

Note we do not need to compare between themselves all N + 1 numbers F k , but 
the first half of them only. The reason is that for any k the classes S k and ZW-fc are 
inversion of each other: En-r = — ^k- Since for any configuration s the equality 
F(s) = F(—s) is fulfilled, we obtain that F k = F N _ k for all values of k. Combining 
the cases of even and odd N in one formula we obtain that to find the global maximum 
of the functional (4) it is necessary to find the largest of the values F k , when k < n = 
[N/2]: 

~N~ 

Fo, Fi, .., F n , n = 

3. Without loss of generality the numbers {cii}f can be put in order according their 
increase: 

ai < a 2 < ... < ajv. (5) 

Let us take any k < n. It is easy to see that the scalar product (s, a) reaches its maxi- 
mum inside the class S k when the configuration vector is 

s(fc) = (-1,-1,...,-!, 1, !,..!). (6) 



Indeed, 



N 



(s(fc),a) = ~y^aj + a,i=a-2a k , 



where 



i=l i=k+l 
N k 

a = ^ai, a k = ^ a i> and «o = 0. (7) 

i=l i=l 

Let s be another configuration vector from the class S k for which numbers of neg- 
ative coordinates j\ < ji < ■■■ < jk dose not take the first places. The scalar product 

k 

(s, a) is equal to (s, a) = a — 2 ^ aj it and inequality (s(fc),a) > (s, a) is fulfilled 

i=l 

k k 

since a i < a U for any set of indices {ji , j 2 , ■■■,jk} that differs from {1, 2, ...,k}. 

i=l i=l 

Thus, under the condition of ordering (5), to find the global minimum of the func- 
tional (3) it is necessary to find the largest among the numbers 



F k = (N - 2k) (a - 2a k ) , k = 0, 1, .., n = 



N 
~2 



(8) 



where a and dk are given in Eq.(7). 

4. The initial problem (l)-(2) can be considered as solved: the expressions (6) re- 
strict the set of configurations among which the ground state of the functional (2) should 
be found. To define which configuration is the ground state it is necessary to calculate 
n numbers (8) and find the largest among them. It reminds unclear under which condi- 
tions this or that configuration (6) would be the ground state. If any of them will be the 
ground state or not? It turned out that these questions can be answered. 

Without loss of generality let us suppose that the numbers a, have a special form: 

a.i=oei-t, ai £ [0,1], £ > 0. (9) 

In this presentation the values on are positive numbers from the unit interval, and the 
positive parameter t can take an arbitrary value. It is not difficult to see that from the 
point of view of minimization of our functional an arbitrary set of numbers a, can be 
reduced to the form (9). For example, let us suppose that a,\ < ... < a« < and 
a,N — ax < 1. Then we set ai = ai — a\ and t = \ai\. This means that the numbers 
ai have the form (9). On the contrary, let the initial numbers a, have different signs and 
take on arbitrary values: 5i < ...0 < ... < ajv, and a — max(|fii| , disr) >> 1. Let 
us normalize these numbers dividing them by 2d: ai = di/2a <E [—1/2, +1/2]. It is 
clear that the solution of the problem (1) is the same when we use initial numbers a, 
or normalized numbers a,. The last numbers can be presented in the form (9), if we set 
Oii = a i + I an d £ = h- From our argumentation it follows that the numbers ai can 
always be presented in the form (9). Then the following statement is right (the proof see 
in the Appendix). 

Theorem. When £ increasing from the initial value £ = 0, the ground state sequen- 
tially coincides with the vectors s(fc) (6) in the following order: 

s(Q) -> s(l) -> ... -> s(k - 1) -> s(fc) -> ... -> s (n - 1) -> s (n) . (10) 

The jump of the ground state s(k — 1) — > s(fc) occurs when £ transfers through the 
critical value: 

ffc = *-2** + (*-2fc + 2)a fc . fc = lj2 .. jnj 
k 2(N-2k + l) ' ' ' ' V ' 

N k 

where analogously of Eq.(7) a = a i an< ^ ®k = a i- When t belongs to the 

i=l i=l 

interval [tk, £fe+i], the ground state of the functional is the configuration s(k). 

This theorem generalizes the previous results obtained in The theorem describes 
exhaustively the behavior of the ground state for the problem (l)-(2). Depending on the 
values of external parameters {a;} each of the configurations s(fc), k = 0, 1, n can 
turn out to be the ground state of the functional. For t from the interval [£&, £fe+i] the 
energy of the ground state is the linear function of the parameter t. It can be easily seen 
from the expressions (8) substituting the values in the form (9): 

E k (t) = A k + t-B k , k = 0,l,...,n, (12) 

where up to the factor 1/2 AT we have: 

A k =a-(N -2k)(a-2a k ), B k = (N - 2k) 2 - N. (13) 




Fig. 1. The dependence of the ground state energy Eqs on the parameter t for additive 
matrices of the dimensionality N = 100: the upper one is the arithmetical additive 
matrix, and the lower one is the random matrix (see the body of the text). The values 
Eos for the points tk are marked. 



In Fig.l for N = 100 it is shown how the energy of the ground state depends on the 
parameter t. The upper panel corresponds to the case when the values a, constitute the 
arithmetical progression: on = i/N, i = 1, 2, .., N. On the lower panel the analogous 
plot is shown for a random additive matrix, when on are random numbers from the 
interval [0, 1]. Along the abscissa axis the values of the parameter t are shown, along 
the axis of ordinates we show the energy of the ground state calculated in the points 
tfc (11). The first value of the parameter t for which the energy of the ground state is 
calculated is equal to zero: t = 0. Since both plots are very similar, we analyze only 
one of them; for example the upper one. 

We see that the energy of the ground state is nontrivially depended on the parameter 
t. For small values, t ~ 0, very deep minima correspond to the ground state. Then, when 
t increases, the depth of the global minimum decreases very quickly and it reaches a 
minimal value when t « 2. For these values of t all matrix elements become negative. 
During further increase of t the depth of the global minimum slowly but steadily in- 
creases. It becomes deeper and deeper. Which properties of the energy surface reflect 
non-monotone change of the depth of the global minimum? What properties are respon- 
sible for its minimal depth? For the time being we cannot answer these questions. Using 
formulae (10)-(13) everyone can be certain of universal character of the curves shown 
in Fig.l. 

Up till now we can neither extend these results onto local minima of the functional, 
nor obtain analytical description of other interesting characteristics such as the num- 
ber of different minima, distribution of local minima with respect to their depths and 
distances to the ground state and so on. However, if the ground state is known, these 
characteristics can be studied with the aid of computer simulations. Now we turn to 
presentation of these results. 

3 Computer Simulation 

For given N and {a{\ for each value of t we generated an additive matrix. We did 10 5 
random starts (see Introduction) and obtained the same number of local minima. For 
each minimum we fixed its depth (the energyi?; ), the relative Hamming distance Di 
between the minimum and the ground state and other characteristics. Thus as a result of 
a great number of random trials for each value of t we could estimate: a) the probability 
of random finding of the ground state pas \ b) the deepest of the obtained minimum 
and the distance from it to the ground state; c) the number of different minima K , their 
distribution over energies and distances from the ground state and so on. The parameter 
t was varied from zero up to the maximal value t n . For two dimensionalities N = 100 
and N = 1000 such experiments were done for both arithmetical and random additive 
matrices. 

In Fig. 2 for the arithmetical additive matrix of dimensionality N = 100 the depen- 
dence of some of the listed characteristics on the parameter t is shown. Let us explain 
what the graphs shown on different panels of the figure mean. 

On the upper panel the probability to find the ground state pas is shown. We see 
that in the region of small values of t (t < 1.8), where the depth of the global minimum 
is large, the probability to find the ground state is notably different from zero. On the 




Fig. 2. For arithmetical additive matrix of dimensionality N = 100 the following graphs 
are shown: on the upper panel is the probability to find the ground state; on the next 
panel is the ratio of depth of the deepest found minimum to the depth of the global 
minimum; on the next panel is the relative Hamming distance between the deepest 
minimum and the ground state; on the bottom panel is the number of different energies 
of local minima. 



contrary, in the region of large values of t, where the global minimum becomes rather 
shallow, the probability to find it is equal to zero (it is less than 1CP 5 ). At the same time 
it is not important which configuration s(k) is the ground state. 

Apparently, such behavior of the probability pes is one more confirmation of the 
law, which was theoretically predicted in Q, 0: the deeper minimum, the greater 
probability to find it under the random search. 

For the matrix of dimensionality N = 1000 the behavior of the given characteristic 
is an analogous one. The value of the parameter t for which the probability to find the 
ground state becomes zero, increases up to the value t « 4. 

On the second panel from the top the ratio of the found deepest minimum E m i n to 
the global minimum Eqs > E m i n /Eas > is shown. This ratio takes on a value from the 
interval [0, 1]. At first, while the ground state still can be found, this ratio is equal to 1. 
Then in the region of the values t « 2.3 — 2.5 this characteristic has a sharp downward 
excursion, which soon changes to a steady increasing and tends to 1 asymptotically. 
The minimal value of this characteristic is E nun /EGS ~ 0.85. It shows that in the 
worst case the objective function is 15% less than the optimal value. 

For matrices of dimensionality N = 1000 the behavior of the ratio E m \ n /EQs is 
absolutely analogous. The deepest downward excursion of the graph takes place when 
t ks 4 , and its depth increases noticeably: the minimal value of the ratio is equal 
Emin/Eag ss 0.5. In other words, when the dimensionality of the problem increases 
the found suboptimal solution will be worse comparing with the global minimum. 

Note, that for the large values of t ~ 7 — 8 , when the ratio E m [ n /EQg is close to 
1, the probability to find the ground state as before is equal to 0. The same also takes 
place in the case N = 1000. 

On the second panel from the bottom it is shown how the distance D between 
the deepest local minimum and the ground state depends on the parameter t. (By the 
distance we understand the relative Hamming distance D = (N — abs(s,s'))/2N e 
[0,0.5].) 

At first, while the ground state still can be found this distance is equal to (see 
the beginning of the graph). Then in the interval of the "worst" values of t the distance 
D increases sharply up to the value D max w 0.3 . After that the distance between the 
deepest local minimum and the ground state is stabilized near the value D = 0.2. Let 
us add that for additive matrices of dimensionality N = 1000 suboptimal solution is far 
away from the ground state. This distance is D as 0.4 . 

The general conclusion is as follows: for rather large values of t , when as a result 
of the random search it is possible to find suboptimal solution only, this solution is suf- 
ficiently far from the ground state. However, the ratio of the minima depths E m i n / Eqs 
can be of order of 1 (in Fig. 2 this situation corresponds to the values of t > 4 ). This 
combination of properties is possible only if the energy surface consists of a large num- 
ber of local minima, which depths not strongly differ one from each other and from 
the global minimum. We may conclude, that for large values of t , when elements of 
connection matrix are large negative numbers, the construction of the energy surface is 
as aforesaid. 

On the bottom panel we show the dependence of the number of different energies 
of local minima K on the value of the parameter t. As a rule each energy is many times 



degenerated. To estimate the number of different local minima it is necessary to analyze 
how many different configurations correspond to the same energy. Nevertheless, such 
characteristic as the number of energy levels is also of interest. 

For an arithmetical additive matrix of the dimensionality TV = 100 the maximal 
value of the characteristic K is reached in the region t <~ 3. This maximum is compar- 
atively small, ~ 500. However, it turns out that each energy is many times degenerated, 
and the number of different local minima is an orders of magnitude greater. For a ran- 
dom additive matrix of the same dimensionality the maximal value of the characteristic 
K is equal to tens of thousands (the graph is not presented). 

For the additive matrices of the dimensionality TV = 1000 the general form of the 
graph of the characteristic K is analogous. In this case the maximal value, K max ~ 
4 • 10 4 , is reached in the region t k 5. Since for each t only 10 5 random starts have 
been done, this means that each second start leads the system into new local minimum. 
In other words, for middle values of t the number of local minima is very big. 



4 Discussion and Conclusions 



For additive matrices the method of finding of the global minimum of the quadratic 
binary functional is pointed out. We propose the t -parametrization of additive matrices 
that allows one to get an exhaustive classification for all variants possible for the ground 
state. 

For not great values of t (let us say for t 6 [0,2]) among matrix elements there 
are positive as well as negative ones; or all elements are negative, but they are small in 
modulus. In this case the depth of the global minimum is very big. Here the probability 
to find the ground state in random search is rather high: pas ~ 0.5 — 1.0. It can be 
supposed that in this case the energy surface has a small number of local minima whose 
depths noticeably less then the depth of the global minimum. 

On the contrary, for the great values of t all matrix elements are negative and they 
are big in modulus. In this case it is practically impossible to find the ground state 
with the aid of the random minimization, since the probability to get into the global 
minimum is negligible small. Apparently in this case the energy surface contains very 
large number of local minima that only slightly differ from each other in depths. Here 
the global minimum is only insignificantly deeper than local minima. Varying the value 
of the parameter t it is possible to get over from one type of the energy surface to the 
other one. So, additive matrices are good models for examining the energy surfaces in 
the general case. 

By this time additive matrices for large values of t can be used for testing of new 
algorithms of the quadratic binary minimization. Indeed, on the one hand, with the aid 
of the formulae (10)-(13) the ground state always can be found. On the other hand, for 
large values of the parameter t it is practically impossible to find the ground state with 
the aid of the random minimization. 
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Appendix 

In the beginning of Section 2 it was shown that only one of configuration vectors s(fc) 
(6), k — 0, 1, n — [N/2] can be the ground state. Using the representation (9) of <Zj 
it is easy to obtain Eq. (12) for the energies of s(fc)-configurations: Ek(t) — At + tBk, 
where and Bk are given by Eq.(13). As functions of the parameter t energies Ek{t) 
are straight lines. We have to analyze the behavior of the set {Ek(i}}^. When a straight 
line Ei (t) is lower all other straight lines, the configuration s(l) is the ground state. 

For simplicity we restrict ourselves to the case of evenA = 2n. Let us write down 
the expression (13) in more details: 

Aq = -(N - 1)6 < Ax < ... < A n = a, , 
B = (N - 1)N > Si > ... > B n = -N. 1 ' 

When fc increasing, the free term Ak of the straight line Ek (t) increases monotoni- 
cally. In other words, the intersection of the straight line with ordinate axis rises higher 
and higher. On the other hand, when k increasing the coefficient Bk decreases mono- 
tonically, so that in the end it even becomes negative. For the case N = 6 the typical 
behavior of the set of straight lines {i?fc(£)}o ^ s snown m Fig-3- We use this figure to 
explain how the ground state depends on the parameter t. 

When £ = all the matrix elements are positive and the configuration s(0) = 
(1, ...1) is the ground state. Let us increase t little by little. At first the straight line 



E (t) is lower than all other straight lines. Consequently, s(0) remains the ground state. 
Than for some value of the parameter t the straight line E (t) is intersected by another 
straight line. After that this straight line turns out to be lower than all other straight 
lines. Taking into account the relations (Al) it is easy to see that the first straight line 
that intersects Eo(t) is E\{t) (see also Fig. 3). After this intersection the configuration 
s(l) becomes the ground state. It is the ground state until another straight line intersects 
the straight line E\{t). After that this straight line turns out to be lower than all other 
straight lines. From the aforesaid argumentation it is evident that it will be the straight 
line E2(t) (see Fig. 3). Then the configuration s(2) will be the ground state, and so on. 
It can be shown that if the straight line E/--i(t) is lower than all other straight lines, 
the first straight line that intersects Ek- \ (t ) is Ek (t ) . The intersection takes place in the 
point tk (11) that is the solution of equation Ak-i + t ■ Bk-i = Ak + t ■ Bk- 
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Fig. 3. For random additive matrix of dimensionality N = 6 the straight lines Ek (t) = 
Ak +t ■ Bk are shown for fc = 0, 1, 2, 3 (see the body of the text). 



